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standard* After an introduction, the document is structured around 
five steps, each of which asks a question of standards agencies* Step 
1 asks, "What expectations do people have of the assessment process?" 
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appropriate to invest local and national resources to ensure that 
assessments are fair, valid, and consistent* Step 2 asks, "What 
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question: "Are expectations met without requiring external 
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"What moderation arrangements need to be put in place to meet 
expectations?," suggesting that national standards agencies establish 
either one central national moderation system or a national 
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now meet expectations?" If the answer is No, step 4 should be 
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The New Zealand Qualifications Authority will 
promote improvement in the quality of education and 
training in New Zealand through the development 
and maintenance of a comprehensive, accessible and 
flexible National Qualifications Framework. 



The Authority's mam functions arc to: 

• coordinate all qualifications in post-compulsory 
education and training (from upper secondary to 
degree level) so they have a purpose and 
relationship to one another that the public and 
students can understand 

• set and regularly review standards as they relate to 
qualifications 

• ensure New Zealand qualifications are recognised 
overseas and overseas qualifications are 
recognised in New Zealand 

• administer national examinations, both secondary 
and tertiary 



© New Zealand Qualifications Authority 1992 

All rights reserved. No part of this publication may be 
reproduced by any means without the prior permission of the 
New Zealand Qualifications Authority. 
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FOREWORD 



Holders and users of national qualifications must have confidence that 
different assessors throughout New Zealand have assessed to the same 
standards. 

Moderation is a broad label that covers activities that help to ensure there is 
uniform interpretation and application of standards. 

The main purpose of this booklet is to assist national standards bodies to 
make decisions about external moderation for their units and qualifications. 

Decisions about external moderation options need to recognise that the 
success of the Qualifications Framework depends on co-operation, mutual 
respect, understanding and commitment on the part of all partners - users, 
providers and the Authority. Each must play its role in fostering the 
partnership. 

Legislation gives tertiary institutions the freedom 'to teach and assess 
students in the manner they consider best promotes learning'. This is a 
freedom they practice within the need to maintain 'the highest ethical 
standards and the need to permit public scrutiny to ensure the maintenance 
of those f tandards' as well as the need for general accountability. 

Legislation gives the Qualifications Authority responsibility for ensuring 
there are mechanisms in place to guarantee that assessment procedures are 
fair, equitable, consistent, and in keeping with the required standard. 

The rights and responsibilities of tertiary institutions and the responsibility of 
the Authority require fine balance. 

This booklet does not directly address funding implications. Funding is 
nonetheless a critical consideration when making decisions about 
moderation requirements. Costs will most often be borne by providers, and 
these will usually be passed on to candidates. The Authority has the 
responsibility to reject proposed external moderation systems that have 
undesirable resource implications for providers, candidates or Government. 



INTRODUCTION: Setting the 
scene 



Fair, valid and consistent assessment 

The broad goal for all assessors is to produce assessments that are fair, valid 
and consistent. This requires: 

• appropriate assessment activities, and 

• accurate assessment decisions. 

It is important to note that there are a number of related issues (such as 
providing candidates with fair re-assessment opportunities) which are not 
within the scof>e of this booklet. 

riie tectinical issues 

A valid assessment activity is one which assesses what it sets out to assess, 
and not something else. 

Another way of thinking about validity is that the as.sessment is 'fit for 
purpose' - it is an appropriate Vv^ay to assess the specific learning outcomes 
that are under consideration. 

An assessment activity will be valid if its results accurately represent achieve- 
ments in the knowledge, skills, attitudes and values that are to be assessed. 

A reliable assessment gives results that are a consistent and accurate picture 
of what is measured. In other words, the results can be relied upon. 

Variation 

Any assessment includes a margin of error. Methods of assessment and 
applications of standards will vary. The challenge is to limit the variation to 
acceptable proportions. 

A shoal of fish has been used as a way of explaining the issue. Note that the 
fish represent assessors, not learners. 



Individual fish might be to the left or right, high or low, forward or laggard, 
each following its own path, each with its own motive power. Yet the shoal 
as a whole can veer in one direction or another. There are apparent 
mechanisms at work within each individual which keep it close to its 
neighbour. 

Through these mechanisms, deviant fish are returned to the mainstream. 
The relationship between individuals adjusts constantiy but the shoal 
remains within appropriate bounds and retains a constant, steady 
progression. Some vaiiations are considered acceptable.^ 



This lllustraHon does not mean learners should not always be encouraged to excel by 
achieving beyond expectations. 





Moderation 

Wherever it occurs, moderation of assessment is usually carried out through 
sampling. This involves targe tting selected, representative p^^ints to check out 
the quality of the whole. 

As individual assessors move through the assessment process, the main 
function of moderation activities is to ensure that different applications of 
standards remain within acceptable limits. In other words, moderation 
ensures that assessors remain within the national shoal. 



This boolilet 

To assist standards bodies to make decisions, this booklet has been structured 
in five steps. Each step asks a question: 



Step 1 What expectations do people have of the assessment process? 

Step 2 What activities are already in place to help ensure assessments are 
fair, valid and consistent? 

Step 3 Are expectations met without requiring external moderation? 

Step 4 If not, what moderation arrangements need to be put in place to 
meet expectations? 

Step S Do arrangements now meet expectations? 



Steps one and two are prerequisite questions that need to be answered before 
asking the key question at step three. Depending on that decision, standards 
bodies will stop at step three or continue to step five. 
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STEP! 

What expectations €Sa 
peaple have of the 
assessment process? 

The assessment process covers the whole business of collecting and 
interpreting evidence about what learners have achieved. 

In step 1 the national standards body first identifies the people who will 
make use of reports on learners' achievements in the units under 
consideration. Many of these people will be represented on the standards 
body. 

The standards body must then ask this question: 



QUESTION 

What expectations do the following have of the assessment process? 

• learners? 

• local users? 

• national users? 

• international users? 

• the education conununity? 

• the wider community? 

• government? 

• other specific interest groups? 

These expectations will help to detennine the extent to which it is 
appropriate to invest local and national resources (for example - finances, 
time and energy) into ensuring that assessments are fair, valid and 
consistent. 
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STEP 2 
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What activities are already in 
place to help ensure 
assessments are fairy yalitl and 
consistant7 





Step 2 involves a brief exploration of the overall quality process. 

Even without external moderation, there already are a number of activities 
that support assessors to operate within acceptable limits. 

Unit registration 

When the Authorit}' registers units, it requires each unit to have clearly 
expressed performance criteria. Unit standards must be unambiguous if 
assessors are to produce fair assessments. 

This is, of course, less easy to achieve in general education than in those 
career areas where skills are very specific. 

Accreditation 

Accredited providers are required to have quality management systems of 
policies and procedures that cover: 

the development and evaluation of teaching programmes; 
financial, administrative and physical resources; 
staff selection, appraisal and development; 
student entry; 

student guidance /support systems; 
practical/work-based components; 
assessment; and 
reporting. 

To make sure that the systems are working, providers are also required to 
undertake on-going evaluation of their policies and procedures. 

There are many ways in which the various accreditation requirements enable 
the production of assessments that are fair, valid and consistent, for example: 

• policies on teaching staff wUl ensure there is an appropriate level of 
expertise on assessment practice; 

• assessment procedures will ensure that internal assessments are within 
acceptable national limits. This will mean that providers and work- 
places will carry out internal moderation. ^ 

• In addition to individual assessors, there are two further players in internal moderation: 

candidates - they can assess themselves against the performance criteria, and compare 
self-assessments with their assessors' assessment? Tliey can discuss any disagreements 
with the assessors, in doing so, they contribute to ensuring fair, valid and consistent 
assessment. 

other assessors - where there are several working in the same field, they can also act as 
internal moderators by cross-checking each other's assessments. 

Although internal moderation involves in-house arrangements, individuals from 
outside a provider or work-place can also be invifed to participate. 
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Re-accreditation 

At the time of re-accreditation, providers will be expected to report on suchi 
matters as: 

• moderation and evaluation activities carried out during the period of 
accreditation; 

• any changes in delivery that have been made as a result of the 
moderation and evaluation activities; 

• any further changes which are being considered. 

At the time of re-accreditation, exterrial evaluations are carried out to ensure 
that providers have been meeting u ?ir obligations. 



Audit 

External audits can be used to check 
place and are operating as intended. 



quality management systems are in 
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The linked quality process 



Tlie following dingrnm shows how the activities that have just beon outlined 
torm n linked qualit\' process. 



A provider's 

Quality Meinagement 
System 

will include internal moderation 
procedures for ensuring that 
assessment is fair, valid and consistent 



t 

Evaluation for 
Re-accreditation 

ensures a prcivider has met 
its obligations 



Audit 

checks systems 



Unit registration 


requires cleady defined standards 


Evaluation for 


- 


Accreditation 




ensures a provider has the 




copacitv to deliver unit standards 




i ■ 4- ^ 





Is this enough to meet expectations? 
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STEP 3 

Are expGctations met without 
rcQuiring external moderation'2 

In step 3 tlie national standards body asks a key question; 

QUESTION 

Can there be confidence that the expectations identified in step 1 can be 
achieved through: 

» unit standards with clear, unambiguous performance criteria/ 

• accreditation requirements, 

• providers' internal moderation procedures for achieving fair, valid 
and consistent assessment, 

• re-accreditation requirements and 

• external audit? 

If the answer is YES, there is no need to proceed. The activities in tlie 
quality process have the capacity to deliver thie expectatioiis defined in 
step 1 without needing to add an external moderation option. 

If the answer is NO, then move on to step 4. 
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STEP 4 

What modcratian atrangentGnts 
need ta be put in place to meet 
Gxpeciatians9 




Step 4 is taken if external moderation is required. This step involves a 
national standards body adopting one of two options: 



Either option 1 a centrally established and directed national 
moderation system 

Or option 2 a natiorial moderation system of local networks. 



OPTION 1 

In option 1 the national standards body and /or its agent designs a national 
moderation system. This system will be centrally established and directed. 

The agent of a national standards body could, for example, be any one or 
a combination of the following: 

the Qualifications Authority, 

a national professional association, 

a national industry organisation, 

an individual provider or a consortium of providers, 

a private consultant. 

The standards body will provide the designers with a brief of broad 
instructions. The brief might cover such matters as the function of the system, 
the preferred method of moderation, cost parameters, implementation 
arrangements, and so on. 

Greater detail on these and other moderation matters is provided in an 
Authority publication titled Desigtung a Moderation System. The purpose of 
this booklet is to provide technical assistance to those given responsibility for 
designing moderation systems. 

Unless individuals on a national standards body have some background in 
assessment and moderation practice, members should avoid confronting the 
complexities in that booklet. The design brief should be negotiated with the 
help of an expert who may or may not be the agent who will design the 
moderation system. Qualifications Authority staff are available to assist with 
this. 
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In all situations it is desirable that standards bodies advise the Authority of 
their intentions before the brief is completed. 

When option 1 is adopted, at the time of unit registration the national 
standards body will submit to the Authority information on the design of the 
moderation system. The Authority resen^es the right to reject this on the 
grounds of impracticality or cost, although earlier contact with the Authority, 
as suggested above, should make this unlikely. 

At the time of accreditation, a provider will make a commitment to 
participate in the national moderation system. 

OPTION 2 

In option 2 the national standards body chooses broad moderation criteria. 

When option 2 is adopted, the moderation criteria will be submitted to the 
Au.nority at the time of unit registration. 

Unequivocal moderation criteria can be stated by using the formula that 
follows. 

The standards body selects from the ten criteria provided and combines these 
into a St 'ement. The statement will comprise one, two or three sentences. If 
necessary, the standards body can choose to tag further information. 

An explanation of the criteria is provided in appendix one. 

Sentence 1 

1. There will be external moderation arrangements which 
OPTIONAL: if necessary, select ONE of 

2. are locally based and 

3. are regionally based and 
OPTIONAL: if necessary, select 

4. include user participation. 




Sentence 2 

OPTIONAL: if necessary, select ONE of 

5. There will be a pre-planned programme fur sampling 

6. There vnll be a flexible programme for sampling 
OPTIONAL: if necessary, select ONE, TWO or THREE of 

7. the design of assessment activities 

8. during assessment activities 

9. assessment decisions. 



A standards body can add a third sentence which will enable effective 
sampling to occur. 



Sentence 3 

OPTIONAL: if necessary, add 

10. Where a provider is accredited to offer 

multiple units in (a) at level/s 

(b) , in any one year assessments in 



Jc) of these will be moderated. 



Sentence three is completed by adding three items of information: 

(a) the grouping of units, for example 'history' or 'fisheries'; 

(b) the relevant Framework level or levels, for example '2' or '1-3'; 

(c) the percentage of units to be sampled. 

The following examples show how a moderation criteria statement can be 
developed using the formula. In example a. the standards body selected only 
two criteria whereas example b. is a full three sentence statement. 
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Example a: 

There will be external moderation arrangements which include user 
participation. 

Example b: 

There will be external moderation arrangements which are locally 
based. There vvill be a flexible programme for sampling the design of 
assessment activities and assessment decisions. Where a provider is 
accredited to ^ffer multiple units in history at levels 1-3, in any one 
year assessmei^ts in 33% of these will be moderated. 



Again it is desirable that the standards body seeks advice on its intentions. 
Qualifications Authority staff are available to discuss with standards bodies 
the implications of the criteria they wish to select. 

The criteria are broad statements that provide a basis for action. Individual 
providers will be required to take the criteria and design a suitable local 
moderation network or join or link to an already established network. The 
publication Designmg a Moderation System provides technical assistance on 
how tc do this. 

At th.e time of accreditation, a provider will need to demonstrate that its 
moderation arrangements meet the criteria laid do^^ n by the national 
standards body and are fit for purpose. 
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Summary of the options 



STEP 3 






Either No external 


Or 


External 


moderation 




moderation 



r 



STEP a 



Or 



Either Option 1 

a centrally established 
and directed national 
moderation system 

The national standards body 
and/or its agents design a 
national moderation system 



Option 2 

a national moderation 
system of local networks 

The national standards 
body chooses moderation 
criteria. 



T 



1 11 be *iccroditod «i 
provider indicatos its 
intention to piirticipate 
in this svstom. 



A provider designs or ic^ms 
.1 loco) moderotion 
network. 

lo be accredited a 
provider demonstrates that 
local arrangements meet 
the moderation criteria. 
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Considerations 

Before adopting one of the options, the following points should be 
considered carefully. 

Moderation is no substitute for good assessment. 

While moderation systems might have importance, they must not in any way 
constrain good learning. 

Excessive external moderation requirements would be an inappropriate use 
of valuable resources. If the expectations defined in step 1 are reasonable, 
regular reference to these should prevent inappropriate resource 
commitments. 

Flexibility 

For units up to levels 3 or 4 of the framework, it is important to note one key 
advantage in option 2. 

Many providers will wish to offer courses that include units belonging to 
different national standards bodies. There are serious resource and logistical 
problems if the units within a course require providers to participate in a 
number of different centrally directed moderation systems (as in option 1). 
Option 2 gives providers the flexibility to design local systems that are 
suitable for delivering qualitv' in the range of units they wish to offer. 



IS 
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STEPS 




Da arrangGtuGnts now meet 
GxpGctations ? 

In step 5 the national standards body considers two final questions: 



Question 1 

Does the linked quality process (shown in the diagram that follows) 
including external moderation now meet the expectations identified in 
step 1? 



If the answer is NO, then step 4 should be revisited. 

If the answer is YES, then move on to the second question. 



Question 2 

Are the arrangements for ensuring the consistency of assessment with 
the required standards efficient, resource effective and accessible ? 



This is the question the Qualifications Authority asks when it receives 
information on proposed moderation arrangements. This occurs at the time 
of unit registration. 

If the answer to question two is YES, the moderation arrangements can be 
considered fit for purpose. 

If the answer is NO, step 4 should be revisited. 
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Unit registration 


requires clearly defined standards 


- Evaiuation for 




Accreditation 




ensures a provider has the 




capacitv to deliver unit stiindards 
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A provider's 

Quality Management 
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^ External 
Moderation 
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will include internal moderation 
procedures for ensuring that 
assessment is fair, valid and consistent 
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Assessment and moderation go hand in hand, A pre-requisite for dealing with 
the complexities that arise in setting up systems for tlie moderation of 
assessment is an understanding of the issues in and practice of standards- 
based assessment, These are introduced in a publication titled Beyond the 
Norm?: An Introduction to Standards-based Assessmmt. 

Designing a Moderation System provides technical assistance to those who are 
given responsibility for designing moderation systems. It will be of interest to 
designers of external systems at the national or local level and designers of 
providers' internal moderation systems. 

The two publications described here are available from: 

The Sales Officer 

The New Zealand Qualifications Authority 

POBox 160 

Wellington 

A further publication titled Quality Managemait Systems for Nationally 
Registered Qualifications is being produced. This will explain the concepts that 
link moderation of assessment to the overall quality process. It will highlight 
the importance of partnership. 



A video on the moderation and assessment process zuill be available in February 1993. 
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APPENDIX ONE 



MODERATION CRITERIA 

The following are explanations of some of the moderation criteria presented for 
option 2 in step 4. 

There will be external moderation arrangements 

'External' means that assessments are open to the scrutiny of people from 
outside the provider or work-place where they are conducted. This is likely to 
involve any one or a combination of the following: 

• assessors attending external meetings; 

• an external person visiting assessors; 

• exchanges of assessment material that do not involve face-to-face 
contact. 

Locally based arrangements 

These would involve providers who are located nearby each other. 
Regionally based arrangements 

This would usually involve all providers withu particular geographical areas. 
Where this criterion is used a tagged explanation may be necessary. 

Include user participation 

A suitable representative from industry or whatever other appropriate user 
group will participate in moderation procedures. 

Include a pre-planned programme 

A plan will be provided at the time of accreditation and will be adhered to. 
A flexible programme 

Information on a predetermined plan will not be required at the time of 
accreditation. 

The design of assessment activities 

The panel or moderator will scrutinise the appropriateness of the design and 
content of assessment activities and any marking schedule before they are used. 

Ouring assessment activities 

A moderator works alongside an assessor while assessment decisions are being 
made. (This is usually only possible if visiting moderators are used.) 

Assessment decisions 

This will occur after assessment activities have taken place and will in\'olve 
scrutinising the accuracy of assessment decisions that have been made about 
candidates' work. 



er|c P^? 
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APPENDIX TWO 



GLOSSARY 

M Accreditation (of providers) - a process for ensuring that providers 
have the capacity/ including management of qua lit)', to deliver unit 
standards. 

■ Assessment - a process of collecting and interpreting evidence of 
competence or achievement. 

■ Moderation of assessment - a process for ensuring the consistency 
of assessnmU ivith the required standard. 

■ Performance criteria - statements of competence or achievement 
against zvhich the attainment of outcomes is measured. 

■ Standards-based assessment - assessment lohich is measured 
against unit standards. 

M Certification - documentary evidence that a qualification has been 
au^arded. 

1 Credit (as applied to a unit standard) - a value assigned to a unit 
standard u^hich reflects the relative time and effort required to 
complete its outcomes. 

■ Credit Transfer - a process of transferring credits between courses 
which lead to a nationally registered qualification. 

■ Database of Student Records - a database which contains details of 
units and qualifications completed by individuals. 

■ Delivery - teaching and learning approaches, context and content, 
resources, and range and number of assessments. 

■ Domain - a term describing a specific area of education or training at 
unit level, defined in the NZQA Standard Classification for Units and 
Qualifications. 

M Element/Outcome - the competencies/ achievements which must be 
demonstrated for successful completion of a unit. 

■ Field - a term describing a general area of education or training, 
corresponding to an ISCED field and defined in the NZQA Standard 
Classification for Units and Qualifications (see ^Iso Sub-field and 
Domain). 

■ Sub-field - a term describing a specific area of education or training at 
qualification lei^el, corresponding to an ISCED sub-field and defined in 
the NZQA Standard Classification for Units and Qualifications. 

■ Group AccREorrATiON - accreditation of a provider to offer a number 
of units in a specified field, sub-field or domain up to a specified level. 
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■ General Accreditation - accreditation of a provider to offer all 
National Certificate and National Diplonna units up to level 7 of the 
framework. 

■ Group Accreditation - accreditation of a provider to offer any 
number of units in a specified field, sub-field or domain up to a 
specified level. 

■ Levels - the eight levels of the framework are defined in terms of 
progressive stages of competence/ achievement and complexity in 
units assigned to them. 

■ Moderation of Assessment - a process for ensuring the consistency of 
assessment with the required standard. 

■ National Qualifications Framework - collectively, all nationally 
registered qua lifica Hons and the nationally registered unit standards 
from which they are derived. Implicit is a defined and logical 
relationship between them. 

■ National Standards Bodies - represent all major user groups 
connected with a field, sub-field or domain (for example, health 
sciences) and have responsibility for the development, evaluation and 
endorsement of all units and qualifications in that category. 

■ NZQA Standard Classification for Units and Qualifications - a 
document which defines fields, sub-fields and domains, used for 
naming qualifications and unit standards, and for assistance in 
defining NSB and ITO fields, sub-fields or domains. 

■ Outcome/Element - the competencies/ achievements which must be 
demonstrated for successful completion of a unit. 

■ Performance Criteria - statements against which the attainment of 
elements/outcomes is measured. 

■ Provider - an individual or organisation providing education or 
training. 

■ Qualification - a combination of unit standards which, when 
certificated, completes the educational prerequisite agreed by national 
standards bodies to be appropriate for entry to an occupation, or to a 
course at a higher level. 

■ Quality Audit - a process for ensuring the effective performance of a 
provider's overall systems for the management of quality; a function 
of the Authority. 

■ Quality Management - a process and structure implemented by a 
provider for ensuring that standards required by the Authority and 
natior\al standards bodies are met; a prerequisite for accreditation. 
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■ Accreditation (of providers) - a process for cnsuriuj^ that providers 
have the capacity, iuchuiiug mauai^etueut of quality, to deliver unit 
standards. 

■ Group Accreditation - accreditation of a provider to offer any 
number of units i}i a specified field, sub-fwld or domain up to a specified 
level. 

■ General Accreditation - accreditation of a provider to offer all 
National Certificate and National Diploma units up to level 7 of the 
frameivork, 

■ Uwrr AccREorrATiON - accreditation of a provider to offer a specific 
unit or units. 

■ Moderation of Assessment - a process for ensurinf^ the consistcuci/ 
of assessment xvith the required standard. 

■ Registration of Private Training Establishments - a process for 
ensuring that basic educational and consumer safeguards arc in place. 

■ Registration of Units and Qualifications - a process for ensuring 
that units and qualifications have been evaluated and eiuiorsed by 
national standards bodies as representing an exclusive national 
standard, and so registered with the Authority. 

Recognition of Prior Learning - a process of awarding credits for 
outcomes which have not been certificated in terms of the National 
Qualifications Framev/ork and may have been completed outside 
formal education and training. 

E Credit Transfer - n process of transferring credits betxveen courses 
which lead to a nationally registered qualification. 

Registration of Private Training Establishments - a process for 
ensuring that basic educational and consumer safeguards are in place. 

Registration of Units and Qualifications - a process for ensuring 
that units and qualifications have been evaluated and endorsed by 
national standards bodies as representing an exclusive national 
standard, and so registered with the Authority. 

Standards - nationally registered statements of education and 
training outcomes and their associated performance criteria. See also 
Unit. 

Standards-based Assessment - assessment which is measured against 
unit standards. 
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Unit - a unit has two parts: 

• natiomlly nmlmtcd and endorsed unit standards registered on the 
Authoritif's unit standard database 

• delivcnj details developed by a provider for teaching purposes, 

■ Delivery - teaching and learning approaches, context and content, 
resources, and range and number of assessments. 

■ Element/Outcome - the competencies/achimmcnts which must be 
demonstrated for successfid completion of a unit. 

■ Unit Standards - nationally registered element/outcome statements 
and performance criteria, and administrative information. 

Unit Accreditation - accreditation of a provider to offer a specific 
unit or units. 

UnitSianoards - nationally registered element/outcome statementb 
and perfoi-mance criteria, and administrative information. 

User Group • all those who derive direct or indirect benefit from a 
qualification and its component units; generally includes students, 
qualified individuals, professional associations and employers who 
may be represented, along with providers, in national standards body. 
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